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Distributed,  testing  is  the  method  of  linking  various  Live ,  Virtual \  and  Constructive  (LVC) 
sites  and  capabilities  together  to  conduct  the  Test  and  Evaluation  (T&E)  of  a  system  or  system- 
of-systems  in  a  distributed  environment.  This  is  normally  done  in  lieu  of  a  large-scale  “ open  air” 
test  using  actual  live  operational  hardware  for  all  systems  involved.  Conducting  distributed 
testing  complements  live-only  testing  and  provides  the  means  for  rapid  integration  of  systems 
early  in  their  developmental  life  cycle.  It  also  provides  an  efficient  means  of  adding  realism  to 
T&E  by  providing  system  representations  not  otherwise  available,  and/or  enabling  interrelated 
systems  not  otherwise  available,  in  realistic  numbers.  Distributed  testing  enhances  the  cross- 
flow  of  test  data  between  T&E  agencies  and  allows  for  the  early  integration  of  Operational 
Test  (OT)  influence  into  Development  Test  (DT).  Conducting  distributed  test  events  will  save 
acquisition  and  T&E  programs  time  and  money,  as  well  as  reduce  risk.  It  can  be  used  to  develop 
the  operationally  representative  Joint  Mission  Environment  (JME)  that  can  be  used  to 
evaluate  the  interoperability  capabilities  of  a  system  or  system-of-systems  in  preparation  for 
“Net-Readiness”  certification.  This  distributed  JME  can  be  developed  at  a  fraction  of  the  cost  of 
live  open  air  scenarios  and  provides  the  capability  to  evaluate  technical  and  operational 
performance  for  individual  systems  and  systems-of-systems  in  realistic  environments.  Even 
with  the  obvious  benefits,  the  concept  of  distributed  testing  is  only  very  slowly  gaining 
recognition  and  acceptance  from  acquisition  program  managers  and  the  T&E  community.  This 
article  examines  the  challenges  of  conducting  distributed  testing  and  provides  an  update  on  what 
is  being  done  to  mitigate  those  challenges  and  to  ensure  success  for  programs  electing  to  take 
advantage  of  the  potential  of  distributed  testing  methodologies. 

Key  words:  Constructive  T&E;  data  repository;  distributed  live;  joint  mission 
environment;  joint  system  effectiveness;  net-readiness;  operation  effectiveness;  system- 
of-systems;  virtual. 


“Distributed  LVC  T&E  is  here,  but  it’s  not 
being  used.’  (Rear  Admiral  Bill  McCarthy, 
USN — retired) 


Defined  policies  and  guidance  for 
conducting  distributed  test  have  been 
in  place  for  quite  some  time.  The 
2004  Department  of  Defense  (DoD) 
Strategic  Planning  Guidance  (SPG) 
for  Joint  Testing  in  Force  Transformation  (DoD 
2004a)  highlighted  the  development  and  fielding  of 


joint  force  capabilities  requiring  adequate  and  realistic 
T&E  in  a  joint  operational  context.  To  do  this,  the 
SPG  recommended  that  the  DoD  provide  new  testing 
capabilities  and  institutionalize  the  evaluation  of  joint 
system  effectiveness  as  part  of  new  capabilities-based 
processes.  As  a  result  of  this  SPG,  the  Director, 
Operational  Test  and  Evaluation  (DOT&E),  was 
tasked  to  develop  the  DoD  Testing  in  a  Joint 
Environment  Roadmap  (DoD  2004b).  This  document, 
approved  by  the  Deputy  Secretary  of  Defense  in 
November  2004,  states  that  current  test  planning 
processes  must  be  updated  and  expanded  to  “clearly 
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indentify  needs  for  adequate  testing  of  joint  warfight¬ 
ing  systems-of-systems  in  their  mission  environment.” 
The  roadmap  also  states  that  “today’s  limited  avail¬ 
ability  of  forces  to  support  T&E  will  be  compounded 
when  joint  mission  capabilities  are  tested  in  assigned 
mission  environments.”  Further,  “a  persistent,  robust 
modern  networking  infrastructure  for  systems-of- 
systems  engineering,  Developmental  T&E  (DT&E), 
and  Operational  T&E  (OT&E)  must  be  developed 
that  connects  distributed  LVC  resources,  enables  real¬ 
time  data  sharing  and  archiving,  and  augments  realistic 
OT&E/Initial  OT&E  (IOT&E)  of  joint  systems  and 
systems-of-systems”  (DoD  2004b). 

A  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
released  in  2008  defines  the  net-ready  Key  Perfor¬ 
mance  Parameter  (KPP)  as  a  mandatory  element  in  the 
complete  life  cycle  of  DoD  systems  to  include  the 
developmental  phase  and  testing  process  (CJCSI 
2008).  CJCSI  6212. 01E  states  that  it  is  Joint  Staff 
policy  to  ensure  that  “DoD  components  develop 
acquire,  deploy,  and  maintain  systems  that  (1)  meet 
the  essential  operational  needs  of  U.S.  forces;  (2)  are 
interoperable  with  existing  and  proposed  standards, 
defined  interfaces,  modular  design;  (3)  are  supportable 
over  the  existing  and  planned  global  information  grid 
(GIG);  and  (4)  are  interoperable  with  allies,  coalition 
partners  and  other  U.S.  and  local  agencies  as 
appropriate.”  CJCSI  6212. 0E  further  states  that 
“...testing  will  verily  the  operational  effectiveness  of 
the  information  exchanges  of  the  system  under  test 
with  all  its  enabling  systems.”  (CJCSI  2008). 

An  argument  in  support  of  distributed  test  can  also 
be  found  in  a  recent  memorandum  by  DOT&E  to  the 
commanders  of  the  Service  Operational  Test  Agencies. 
The  memorandum  says,  in  part, 

“Thus,  operational  effectiveness  and  suitability 
must  be  evaluated  and  reported  on  the  basis  of 
whether  a  system  can  be  used  by  soldiers,  sailors, 
airmen,  and  Marines  to  accomplish  a  combat 
mission.  The  appropriate  environment  for  that 
evaluation  includes  the  system  under  test  and  all 
interrelated  systems  ( that  is,  its  planned  or 
expected  environment  in  terms  of  weapons, 
sensors,  command  and  control,  and  pla  for ms,  as 
appropriate)  needed  to  accomplish  an  end-to-end 
mission  in  combat.  The  data  used for  evaluation 
are  appropriately  called  measures  of  effectiveness, 
because  they  measure  the  military  effect  (mission 
accomplishment)  that  comes  from  the  use  of  the 
system  in  its  expected  environment.  This  state¬ 
ment  of  policy  precludes  measuring  operational 
effectiveness  and  suitability  solely  on  the  basis  of 
system-particular  performance  parameters.  ” 


The  DoD  policies  outlined  above  require  that  joint 
interoperability  and  net-readiness  testing  be  conducted 
during  the  acquisition  and  fielding  process  for  new 
systems.  Satisfying  the  interoperability  and  measures  of 
effectiveness  requirements  stated  above  as  well  as  net- 
ready  KPP  compliance  will  require  the  testing  of  the 
interactions  of  multiple  systems  at  the  same  time.  This 
will  then  present  a  program  manager  with  three 
options: 

1.  Use  conventional,  live,  open  air-only,  T&E 
methodologies  to  provide  a  representative  and 
realistic  joint  operational  environment.  This  usu¬ 
ally  requires  building  a  large  and  expensive  T&E 
event  that  is  subject  to  limiting  factors  such  as 
low  density/high  demand,  real-world  operational 
priorities  and  are  generally  not  co-located  at  the 
desired  T&E  venue.  Relying  solely  on  live 
systems  is  often  impractical,  usually  too  expen¬ 
sive,  and  sometimes  simply  impossible. 

2.  Use  a  Modeling  and  Simulation  (M&S)-only 
methodology  with  no  live  systems.  In  this  case, 
regardless  of  how  well  a  model  or  constructive 
simulation  is  developed,  it  will  be  very  difficult  to 
garner  credibility  on  how  the  System  Under  Test 
(SUT)  will  function  in  the  real  world.  An  M&S- 
only  T&E  strategy,  without  at  least  some  influence 
of  the  expected  operating  environment  will  simply 
not  be  acceptable  by  the  approvers  of  the  program’s 
Test  Evaluation  Master  Plan  or  the  operational 
forces  that  will  be  required  to  use  the  system.  For 
credible  T&E,  there  must  be  some  level  of 
influence  from  a  real  system  and/or  a  real  operator. 

3.  Use  a  mixture  of  live,  Hardware-In-the-Loop 
(HWIL),  virtual  simulations,  and  constructive 
capabilities  in  a  distributed  environment.  That  is, 
connect  the  various  testing  components  to  form  a 
distributive  operating  environment  for  the  SUT, 
linking  all  its  enabling  systems.  In  this  distributed 
environment,  the  testing  components  and  sys¬ 
tems  need  not  be  co-located.  This  distributed  test 
approach  will  allow  the  program  manager  to 
customize  the  T&E  methodology  to  capitalize  on 
the  particular  advantages  of  each  capability  as  it  is 
needed.  Capitalizing  on  a  distributed  test  infra¬ 
structure,  testing  components,  and  systems  not 
previously  available  can  now  be  fully  integrated 
early  and  continuously  in  the  developmental  and 
T&E  process.  Each  test  event  need  not  be  solely 
large  scenario-driven  events  or  incorporate  the 
available  LVC  assets  all  at  once.  One  event  may 
include  live  and  virtual  capabilities,  another  may 
include  virtual  and  constructive,  while  a  third 
may  link  multiple  HWIL  facilities  with  a  live 
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operator.  Smaller-scale  testing  can  be  done  using 
a  distributed  infrastructure  to  provide  technical 
risk  reduction  prior  to  linking  the  environment  of 
all  interrelated  systems  in  a  larger  test.  Using 
distributed  test  capabilities  may  prove  to  be  the 
simplest,  quickest,  and  cheapest  way  to  avoid  the 
pitfall  of  “measuring  operational  effectiveness  and 
suitability  solely  on  the  basis  of  system-particular 
performance  parameters.”3 

The  complexity  and  expense  of  today’s  military 
systems  clearly  demonstrate  the  need  to  test  early  and 
often  throughout  the  development  and  fielding  process. 
Early  testing  of  a  system’s  capability  to  operate  in  its 
intended  environment  will  allow  designers  and  system 
engineers  to  identify  and  correct  fundamental  issues 
with  performance  and  interoperability  before  they 
become  operational  specifications.  As  the  transition  to 
realize  net-centric  warfare  is  accelerated,  the  require¬ 
ment  to  successfully  demonstrate  systems  interoperabil¬ 
ity  will  increase.  Thus,  the  need  to  use  distributed  test  in 
a  joint  mission  environment  will  increase. 

That  being  said,  how  can  program  managers  and  the 
rest  of  the  T&E  community  be  convinced  to  make  use 
of  the  advantages  of  distributed  test?  What  is  the 
process?  What  are  the  challenges  of  distributed  test? 
What’s  being  done  to  mitigate  those  challenges?  How 
does  a  range  or  facility  begin  to  transition  to 
distributed  test?  While  the  concept  of  distributed  test 
has  not  yet  fully  caught  on,  it  is  my  belief  that  the 
T&E  community  is  in  the  walking  phase  just  prior  to 
beginning  a  run.  It  is  my  intent  to  address  each  of  these 
questions  in  an  effort  to  quicken  our  pace  a  bit. 

What  is  the  process  for  conducting 
distributed  test? 

In  December  2005,  the  DoD  directed  the  develop¬ 
ment  of  the  Joint  Mission  Environment  Test  Capa¬ 
bility  (JMETC)  Program  to  provide  the  test  infra¬ 
structure  necessary  for  conducting  joint  distributed  test 
events  by  cost-effectively  integrating  LVC  test  re¬ 
sources  configured  to  support  the  users’  specific  needs 
for  each  event.  JMETC  was  placed  under  the  Office  of 
the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  (OUSD  [AT&L]),  with 
responsibility  for  execution  assigned  to  the  Director, 
Test  Resource  Management  Center  (TRMC).  In 
October  2006,  the  JMETC  Program  Management 
Office  was  established  under  the  TRMC.  Conducted 
during  the  summer  of  2007,  Integral  Fire  07 
represented  the  inaugural  use  of  the  JMETC  infra¬ 
structure  to  formally  support  a  distributed  test.  Ten 
months  after  JMETC’s  inception,  the  JMETC  net¬ 
work  was  established  at  five  locations,  and  JMETC 


assisted  in  successfully  linking  19  sites  across  three 
network  enclaves  using  the  United  States  Joint  Forces 
Command  (USJFCOM)  Joint  National  Training 
Capability  (JNTC)-sponsored  Network  Aggregator  at 
Patuxent  River.  JMETC  was  a  major  contributor  to 
Integral  Fire  07  providing  technical  capabilities, 
infrastructure  management,  and  technical  assistance 
in  test  event  planning,  preparation,  and  execution  (Test 
and  Training  Enabling  Architecture  [TENA]  2008). 
In  their  After  Action  Report,  the  Integral  Fire  07  staff 
reported  that  they  were  100%  effective  in  data 
collection  and  all  test  objectives  were  met. 

The  Testing  in  a  Joint  Environment  Roadmap  (DoD 
2004b)  recognized  the  need  to  expand  beyond  the  single 
system  T&E  environment  to  the  distributed  joint 
mission  environment,  so  the  TRMC  assigned  JMETC 
the  mission  of  providing  a  persistent  capability  for 
linking  distributed  facilities,  therefore  enabling  DoD 
customers  to  develop  and  test  warfighting  capabilities  in 
a  joint  context.  As  such,  JMETC  provides  the  DoD 
T&E  community  with  the  persistent  network  infrastruc¬ 
ture  (network,  integration  software,  tools),  as  well  as  the 
resident  distributed  test  expertise  needed  for  the 
connection  and  use  of  distributed  LVC  resources  to 
support  the  DT&E  and  OT&E  of  joint  systems  and 
systems-of-systems.  JMETC  now  provides  a  DoD- 
wide  capability  for  the  T&E  of  a  weapon  system  in  a 
joint  context  to  include  DT,  OT,  interoperability 
certification,  net-ready  KPP  compliance  testing,  and 
joint  mission  capability  portfolio  testing.  This  corporate, 
persistent,  and  reusable  capability  avoids  the  recurring 
cost,  time,  and  effort  that  individual  programs  must 
endure  to  develop  an  expensive  and  temporary  LVC 
infrastructure  for  each  distributed  event. 

To  accomplish  its  mission,  the  JMETC  Program 

•  maintains  a  core  reconfigurable  infrastructure 
that  enables  the  rapid  integration  of  LVC 
resources; 

•  has  integrated  existing  products  that  provide 
readily  available  connectivity  over  existing  DoD 
networks;  standard  data  transport  solutions,  tools, 
and  utilities  for  planning  and  conducting  distrib¬ 
uted  integrations;  and  a  reuse  repository;  and 

•  provides  customer  support,  both  on-site  and  help 
desk,  facilitating  the  use  of  JMETC  to  integrate 
LVC  resources. 

The  JMETC  Program  relies  heavily  on  the  collab¬ 
oration  of  the  Services,  USJFCOM,  and  other  T&E 
agencies  to  build  an  infrastructure  relevant  to  current 
and  future  requirements.  In  order  to  facilitate  and 
formalize  this  exchange  process,  the  JMETC  Program 
Office  instituted  the  JMETC  Users  Group.  The 
JMETC  Users  Group  is  composed  of  technical  and 
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management  representatives  from  acquisition  program 
offices,  T&E  organizations  and  programs,  HWIL 
facilities,  virtual  simulation  facilities,  and  other  labo¬ 
ratories  and  ranges.  These  are  the  JMETC  customers, 
potential  customers,  network  providers,  and  tool 
developers  of  the  JMETC  infrastructure  and  products. 
The  Users  Group  focuses  on  technical  requirements 
and  solutions  and  makes  recommendations  to  resolve 
technical  issues  associated  with  distributed  test.  This 
focus  includes  improving  integration  capabilities, 
connectivity  and  modernization  issues,  middleware 
and  objects  model  requirements,  and  change  coordi¬ 
nation.  The  JMETC  Users  Group  also  provides  a 
forum  for  customers  to  outline  their  customer  support 
requirements.  In  addition,  the  JMETC  Users  Group 
performs  the  important  function  of  consolidating  the 
requirements  of  the  distributed  test  community  and 
making  recommendations  to  improve  JMETC  pro¬ 
cesses  and  procedures. 

JMETC  customers  have  discovered  that  a  standout 
benefit  of  this  capability  is  the  cost  and  time  savings. 
An  example  of  the  benefit  of  using  JMETC’s 
capabilities  comes  from  the  U.S.  Air  Force  (USAF) 
Joint  Expeditionary  Force  Experiment  (JEFX).  JEFX 
is  a  Chief  of  Staff  of  the  Air  Force-directed  series  of 
experiments  that  combine  LVC  forces  to  create  a  near¬ 
seamless  warfighting  environment  in  order  to  assess 
the  ability  of  selected  initiatives  to  provide  needed 
capabilities  to  warfighters.  JEFX  initiatives  are  new 
operational  concepts  and  technologies  designed  to 
close  capability  gaps  and  provide  means  to  satisfy 
warfighter  requirements.  JMETC  has  been  a  support¬ 
ing  partner  for  JEFX  since  early  2008.  The  assessment 
of  multiple  initiatives  using  the  distributed  methodol¬ 
ogies  employed  by  JEFX  is  one  of  the  major  successes 
of  the  program  and  contributes  to  the  overall  success  of 
JEFX  (TENA  2010).  The  After  Action  Report  for 
JEFX  2009  reported  that 

“The  JMETC  network  was  successfully  used  and 
promises  to  become  an  effective ,  persistent 
network  for  tests  and  experimentation.  Overall 
communications ,  networks  and  data  links  were 
very  stable ,  'reach  back’  to  Langley  and  technical 
support  was  successful.  ” 

The  cost  savings  to  JEFX  by  using  the  JMETC 
Infrastructure  and  support  was  reported  at  $4.0 
million,  and  the  recommendation  was  made  to 
integrate  other  USAF  T&E  capabilities  “unto  the 
JMETC  Network.”4 

Also,  as  part  of  JEFX  2010,  the  Spirit  Integrated 
Collaborative  Environment  (ICE)  test  event  provided 
an  assessment  of  the  Link  16  interoperability  for  the 


Air  Force  B-2  and  Airborne  Warning  and  Control 
System  (AWACS).  Using  a  distributed  network 
provided  by  JMETC,  the  assessment  included  a 
mission-ready  B-2  aircraft  parked  on  the  ramp, 
manned  by  a  mission-ready  crew  (not  test  pilots)  and 
was  conducted  at  a  fraction  of  the  cost  of  a  live  open  air 
range  event.  The  distributed  nature  of  Spirit  ICE 
allowed  the  assessors  to  investigate  digital  target 
transfer  capability  and  situational  awareness  in  a 
realistic,  time-sensitive  targeting  scenario  using  tactical 
command  and  control. 

While  cost  and  time  savings  are  certainly  significant 
factors  in  adopting  JMETC  as  a  joint  distributed  test 
solution,  a  third  key  benefit  is  risk  reduction.  JMETC’s 
unique  total  package  capability  allows  the  T&E 
customer  to  minimize  the  technical  risk  associated 
with  planning  for  and  providing  the  distributed  test 
infrastructure  in  order  for  the  focus  to  truly  remain  on 
test  requirements.  JMETC  support  includes  experi¬ 
enced  and  highly  skilled  distributed  test  experts  who 
are  forward  deployed  for  distributed  planning  and 
operations;  a  modern,  tested,  and  reliable  network 
already  in  place;  and  data  exchange  solutions  that  have 
already  been  tested,  proven,  and  put  into  practice. 
JMETC  is  also  the  T&E  community’s  enterprise-level 
focal  point  for  collecting  and  maintaining  “lessons 
learned”  and  implementing  resource  “reuse”  repository 
for  improving  the  DoD  distributed  test  capability. 
Distributed  test  lessons  learned  and  other  important 
support  information  are  available  free  of  charge  online 
at  the  JMETC  Reuse  Repository  (www.jmetc.org).5 
JMETC  actively  captures  customers’  needs  and 
requirements  on  a  continuous  basis  from  program 
planning  through  distributed  event  execution  and 
provides  the  full  support  needed  for  successful 
distributed  test  events  (Figure  1).  For  this  reason, 
program  managers  should  consider  contacting  JMETC 
to  investigate  if  distributed  test  is  a  valid  solution  for 
their  testing  requirements. 

What  are  the  challenges  for 
implementing  distributed  test? 

Historically,  acquisition  program  managers  and  the 
T&E  community  at  large  have  been  doubtful  of  the 
value  distributed  test  brings  to  their  test  programs. 
This  doubt  or  lack  of  acceptance  is  because  they  see 
distributed  test  as  bringing  risk  to  their  program  rather 
than  reducing  risk.  There  is  also  an  underlying 
perception  that  they  will  “lose  control”  of  their  test 
and  their  data.  Some  may  believe  distributed  test,  using 
LVC  assets,  increases  the  risk  to  their  system’s 
performance  as  well  as  their  overall  program  schedule 
and  budget.  Therefore,  some  program  managers  are 
hesitant  to  take  advantage  of  the  benefits  of  distributed 
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Available  JMETC  Support 

You  worry  about  the  test— we  worry  about  the 
infrastructure 
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Figure  1.  Support  given  by  Joint  Mission  Environment  Test  Capacity  (JMETC)  to  the  Department  of  Defense  distributed  test 
community.  JMETC  provides  infrastructure  support  based  on  the  requirements  provided  by  the  program  office.  /As  a  fundamental  part 
of  its  support,  JMETC  assists  in  distributed  test  planning,  provides  the  necessary  test  support  tools,  manages  the  network,  and 
provides  the  middleware  and  technical  support  necessary  for  customers  to  conduct  distributed  test  events. 


test  or  to  incorporate  distributed  test  in  their  test 
planning  process. 

However,  the  reality  is  that  the  proper  use  of 
distributed  test  will  require  less  hardware,  less  time, 
and  less  money  to  conduct  T&E  of  a  system  earlier  in 
its  acquisition  timeline  and  in  its  appropriate  operating 
environment.  This  will  allow  program  managers  to  fix 
problems  earlier  and  with  less  cost.  As  noted  before, 
not  all  distributed  test  events  will  need  to  be  large-scale 
and  expensive  events.  JMETC  has  supported  a  variety 
of  large  and  smaller  scale  distributed  test  events.  A 
sampling  of  significant  events  includes: 

•  Future  Combat  Systems  (FCS,  the  precursor  to 
the  Army’s  Brigade  Combat  Team  Moderniza¬ 
tion  Program)  Joint  Battlespace  Dynamic  Decon- 
fliction  (JBD2)  in  2008.  The  event  was  a 
significant  effort  designed  to  assess  the  readiness 
of  FCS  test  technologies  in  preparation  for  its 
Milestone  C  test  activities.  JMETC  provided  the 
persistent  network  connectivity,  software  inter¬ 
faces,  and  software  tools  needed  to  support  the 
test.  For  the  event,  JMETC  connected  16 
laboratories  with  over  60  applications  from  U.S. 
Army,  Air  Force,  Navy,  and  Marine  sites.  Other 
network  enclaves  integrated  into  the  JMTEC 


network  via  the  Aggregation  Router  included  the 
Air  Force-ICE  (AF-ICE)  and  the  Joint  Training 
and  Experimentation  Network  (JTEN).  During 
JBD2,  JMETC  demonstrated  the  ability  to 
support  a  customer’s  design,  integration,  devel¬ 
opment,  and  execution  of  a  large  and  complex 
joint  distributed  test. 

•  F/A-18  Interoperability  Check.  When  personnel 
from  Naval  Air  Station  (NAS)  China  Lake 
attended  a  JMETC  Users  Group  in  May  of 
2008,  they  were  introduced  to  the  program  and 
learned  which  sites  were  available.  Following  the 
users  group,  the  data  link  testers  called  the 
JMETC  Program  Manager  and  requested  sup¬ 
port  linking  the  F/A-18  lab  at  China  Lake, 
California,  with  the  F-16  lab  at  Eglin  Air  Force 
Base  (AFB),  Florida.  Within  2  days,  after 
verifying  ports  and  protocols  were  open  at  site 
firewalls,  JMETC  announced  the  infrastructure 
was  ready,  and  2  days  later  the  first  interopera¬ 
bility  test  between  the  systems  was  completed. 
The  initial  test  identified  problems,  so  the  F/A- 
18  lab  completed  software  modifications  that  day, 
and  a  successful  re-test  was  accomplished  the 
following  day.  One  week  from  initial  coordina¬ 
tion  to  a  completed  distributed  test  and  solution! 
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•  The  Joint  Surface  Warfare  (JSuW)  Joint  Capa¬ 
bility  Technology  Demonstration  (JCTD)  spon¬ 
sored  by  USN  NAV/AIR  PMA-201  on  behalf  of 
U.S.  Pacific  Command  (USPACOM)  evaluated 
Joint  Surface  Warfare  Net-Enabled  Weapons 
(NEW)  and  the  third-party  targeting  process. 
The  JSuW  JCTD  integrated  Link  16  Jll 
message  set  into  existing  software  for  the 
JSTARS  and  Littoral  Surveillance  Radar  System 
(LSRS)  to  ensure  interoperability  with  the  Joint 
Stand-Off  Weapon  (JSOW-C-1)  and  the  F/A- 
18  E/F,  as  well  as  other  weapons  delivery 
platforms  and  command  and  control  assets.  This 
event  executed  a  full  week  of  high  fidelity  man- 
in-the-loop  tests  on  time  and  with  all  test 
objectives  met  and  is  a  prime  example  of  early  risk 
reduction.  The  JMETC  infrastructure  was  able  to 
peer  a  virtual  F/A-18E/F  Low  Cost  Trainer  at  St. 
Louis  (Boeing)  with  a  virtual  P-3  LSRS  at 
McLean  (MITRE)  and  a  virtual  joint  surveillance 
and  target  attack  radar  system  (JSTARS)  at 
Melbourne  (Northrop  Grumman).  For  the  2009 
JSuW  events,  DoD  and  industry  sites  on  disparate 
networks  were  connected  to  support  the  event 
requirements  with  an  “idea-to-execution”  schedule 
of  only  3  months. 

The  Secretary  of  the  Air  Force  Modeling  and 
Simulation  Policy  Division  (SAF/XCDM)  has  signed 
a  support  agreement  that  incorporates  JMETC  into 
the  event  planning  process  as  an  integration  solution 
for  Air  Force  Distributed  LVC  event  requirements. 
As  of  October  1,  2009,  JMETC  has  assumed 
infrastructure  support  responsibility  for  distributed 
LVC  tests  and  events  sponsored  by  the  Air  Force.  Air 
Force  partners  using  AF-ICE  will  depend  on 
JMETC  to  provide  a  persistent  and  dependable 
distributed  infrastructure.  The  Army  Cross  Com¬ 
mand  Collaboration  Effort  (3CE)  is  developing  a 
plan  to  move  to  the  JMETC  Infrastructure  over  a  3- 
year  period  and  the  Navy  Distributed  Engineering 
Plant  (DEP)  is  developing  a  plan  to  partner  with 
JMETC. 

The  message  to  program  managers  is  that  JMETC 
is  operational  now  and  provides  the  infrastructure 
necessary  for  distributed  test  available  today.  In 
supporting  these  and  others  programs  and  events, 
JMETC  has  established  a  track  record  of  responsive 
support  using  a  reliable  infrastructure.  In  the  three  and 
one-half  years  since  Integral  Fire  07,  the  JMETC 
infrastructure  has  grown  to  over  50  sites  and  is  being 
used  every  week.  JMETC  has  demonstrated  the 
capability  to  support  multiple  T&E  customers  during 
a  single  distributed  test  event,  as  well  as  the  ability  to 


move  large  amounts  of  data  in  support  of  T&E 
requirements.  The  bottom  line  is  that  JMETC  saves 
T&E  programs  time  and  money. 

Program  managers  and  test  directors  can  use 
distributed  test  through  the  JMETC  infrastructure  to 
take  advantage  of  more  frequent,  smaller  events,  and 
even  one-on-one  systems  interoperability  tests  as  well 
as  large-scale  scenario-based  testing.  Distributed  test 
can  be  a  mix  of  various  combinations  of  LVC  assets 
and  capabilities.  Having  the  persistent  infrastructure 
capability  that  JMETC  provides  will  allow  program 
managers  to  find  problems  early  in  the  system’s 
developmental  cycle — when  they  are  cheaper  and  easier 
to  fix.  The  use  of  smaller  scale  and  frequent  distributed 
test  events  will  allow  T&E  programs  to  test  earlier  and 
test  more  often  in  a  program’s  life  cycle.  Operational 
testers  will  be  able  to  leverage  appropriate  develop¬ 
mental  test  data  and  provide  an  early  operational 
influence  to  a  systems  development.  As  stated  by  the 
Director,  Operational  Test  &  Evaluation,  Deputy 
Director  for  Net-Centric  and  Space  Systems,  “There 
is  not  enough  time  to  wait  until  the  end  of  a  program 
to  find  out  what’s  wrong.”6  All  this,  in  turn,  will  have  a 
transformational  affect  on  the  ability  of  program 
managers  to  field  systems  that  are  truly  interoperable 
and  net-ready  for  operational  users — quicker,  cheaper, 
and  at  less  risk! 

It  is  true  that  there  are  technical  challenges  to 
employing  distributed  test.  Various  technical  issues 
have  been  identified  and  resolved  through  the  JMETC 
Users  Group.  Other  issues  continue  to  be  worked. 
Some  of  those  challenges  include  difficulties  in 
satisfying  the  DoD  Information  Assurance  Certifica¬ 
tion  and  Accreditation  Process  (DIACAP);  ensuring 
that  infrastructure  is  capable  of  transporting  large 
amounts  of  data  and  within  acceptable  latency  limits; 
and  addressing  the  requirements  for  Multi-Level 
Security  (MLS)  and  cross-domain  solutions  (i.e.,  how 
different  classification  levels  are  to  be  integrated  into  a 
common  infrastructure,  or  how  to  pass  data  between 
domains  without  compromising  security). 

However,  significant  progress  has  been  made  in 
resolving  these  and  other  technical  issues.  By  far  the 
most  critical  challenge  for  distributed  test  is  misper¬ 
ception.  The  perceived  risk  of  the  distributed  test 
process  and  infrastructure  is  the  determining  factor 
that  must  be  overcome  in  order  to  convince  program 
managers  and  other  T&E  agencies  to  embrace  the 
concept  of  distributed  test.  “We  are  working  the 
engineering  element  to  make  distributed  test 
work... The  hard  part  is  solving  the  human  ele¬ 
ment.”7  If  we  can  solve  the  issue  of  hesitancy  with 
distributed  test,  we  can  solve  the  rest  of  the  technical 
issues  as  well. 
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What  is  being  done  to  mitigate  the 
implementation  challenges? 

The  JMETC  Program  is  actively  engaged  in 
changing  the  paradigm  of  distributed  test.  The  prime 
targets  are  the  acquisition  program  managers  and  the 
T&E  community.  JMETC  has  an  aggressive  outreach 
program  used  to  cause  testers  to  think  about  distrib¬ 
uted  test.  Participating  in  conferences,  briefings  to 
project  managers  and  DoD  Senior  Leaders,  and  articles 
(such  as  this  one)  help  to  advocate  the  advantages  of 
distributed  test  to  the  acquisition  test  community. 
However,  JMETC  is  also  involved  in  solving  the 
technical  problems  associated  with  distributed  test. 
JMETC  recently  completed  an  effort  leading  a  tiger 
team  to  streamline  DIACAP  procedures  in  cooperation 
with  the  Services.  The  tiger  team  recently  completed  its 
work  and  presented  a  list  of  actionable  recommenda¬ 
tions  to  the  DoD  Information  Assurance  community. 
The  Services  are  already  in  the  process  of  implementing 
some  of  the  recommendations.  For  more  details  on  the 
progress  of  the  DIACAP  tiger  team,  go  to  the  JMETC 
Web  site  at  www.jmetc.org.  JMETC  is  also  actively 
working  to  mitigate  multi-level  security  challenges  by 
participating  in  a  Central  Test  and  Evaluation  Invest¬ 
ment  Program  (CTEIP)  project  that  is  designed  to 
mitigate  three  categories  of  MLS  issues  (classification, 
proprietary,  and  coalition).  JMETC  is  providing  data  on 
distributed  infrastructure  performance  that  will  be  used 
to  establish  verification,  validation,  and  accreditation 
guidelines  and  policy.  In  a  recent  test  event  supported  by 
JMETC,  the  U.S.  Navy  Program  Executive  Office, 
Integrated  Weapons  Systems,  Integrated  Combat 
Systems  (PEO  IWS  1.0),  in  conjunction  with  Wallops 
Island  Surface  Combat  Systems  Center  (SCSC)  and  the 
Naval  Surface  Warfare  Center  (NSWC)  at  Dahlgren, 
Virginia,  ran  a  successful  30-hour  stress  test  to  baseline 
Aegis  systems  capability.  This  test  included  measuring 
the  sites’  capacity  to  harvest  massive  amounts  of  data 
over  the  JMETC  infrastructure.  The  PEO  IWS 
requirement  was  to  be  able  to  transmit  200  GB  of  data 
in  8  hours.  With  the  JMETC  infrastructure,  they  were 
able  to  transfer  313  GB  in  8  hours  21  minutes.  Before 
JMETC  connectivity,  PEO  IWS  would  physically 
transport  by  car  the  data  on  hard  drives  between  the 
two  sites.  Due  to  the  success  of  JMETC’s  support,  PEO 
IWS  has  recently  signed  a  formal  support  agreement 
with  JMETC  that  will  allow  them  to  expand  their  use  of 
the  JMETC  infrastructure  to  include  other  sites  to 
further  streamline  post-mission  analysis  of  Aegis  test 
data.  In  accordance  with  the  support  agreement, 
JMETC  will  provide  a  secure  network  for  connectivity 
of  PEO  IWS  test  resources  and  project  development 
efforts.  Also,  JMETC-sponsored  products  will  be  used 
in  building,  sustaining,  and  connecting  the  architecture 


to  test  resources  required  for  PEO  IWS  test  events.  In 
short,  PEO  IWS  will  now  be  able  to  concentrate  fully 
on  their  test  events,  and  JMETC  will  concentrate  on 
their  distributed  infrastructure. 

JMETC  employs  a  dedicated  customer  support  team 
with  extensive  expertise  in  the  JMETC  infrastructure 
and  distributed  testing  in  general  and  will  provide 
guidance  and  assistance  in  the  use  of  the  JMETC 
infrastructure.  To  ensure  a  successful  test,  the  JMETC 
Program  Office  will  assign  dedicated  personnel  to  each 
customer  to  assist  with  planning,  preparation,  integra¬ 
tion,  and  execution  of  the  customer’s  infrastructure 
requirements  for  the  distributed  LVC  test  event.  The 
JMETC  team  is  available  to  support  and  advise  a 
customer  beginning  in  the  early  phase  of  test  planning 
development  and  to  assist  in  developing  distributed  test 
requirements,  alternatives  to  meet  test  requirements  and 
planning  for  network  characterization,  network  config¬ 
uration,  and  connectivity  testing.  The  key  to  success  for 
any  program  in  the  use  of  distributed  test  will  be 
including  distributed  requirements  into  early  test 
planning  documentation.  JMETC  is  available  to  assist 
the  program  in  incorporating  distributed  test  require¬ 
ments  into  test  planning  documents  such  as  Test 
Evaluation  Master  Plans  (TEMPs).  Program  managers 
need  only  contact  JMETC  to  coordinate  for  support  in 
evaluating  potential  distributed  test  strategies  and  then 
documenting  distributed  requirements  into  the  TEMP. 

During  test  execution,  the  JMETC  team  will  be 
available  for  on-site  support.  Test  execution  support 
includes  the  development  of  test  support  tools  and 
training  as  well  as  “on-call”  or  online  technical  support 
and  network  troubleshooting.  JMETC  will  assist  with 
data  logging  and  data  analysis  tools  after  the  test  event, 
and  network  performance  analysis  before,  during,  and/ 
or  after  event  execution.  An  important  aspect  of  the 
JMETC  Program  is  the  sharing  of  infrastructure  and 
distributed  testing  lessons  learned.  The  JMETC  Reuse 
Repository  is  structured  to  give  the  user  community 
easy  access  to  general  program  information,  questions 
and  answers,  lessons  learned,  opportunities  for  distrib¬ 
uted  test  event  collaboration,  and  insight  into  the 
capabilities  of  JMETC  and  other  JMETC  users. 
However,  the  test  program  retains  complete  control 
of  the  data  and  full  control  of  authority  to  release  to 
individuals  and  outside  agencies.  In  addition,  JMETC 
provides  each  of  its  sites  and  customers  the  capability 
of  hosting  their  own  space  on  the  reuse  repository  to 
facilitate  collaboration  for  specific  events,  tools,  or 
sites.  The  JMETC  Reuse  Repository  can  be  found  at 
www.jmetc.org.  Also  available  on  the  Web  site  is  the 
JMETC  Users  Handbook,  which  provides  current  and 
potential  customers  a  working  knowledge  of  the 
JMETC  program  as  well  as  how  to  utilize  JMETC. 
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Conclusion 

While  we  recognize  that  integrating  a  distributed 
requirement  into  test  planning  and  execution  can 
sometimes  be  a  technical  and  cultural  challenge,  the 
T&E  community  is  beginning  to  realize  the  real 
benefits  of  distributed  test.  It  is  a  fact  that  the  use  of 
the  JMETC  persistent  infrastructure  lowers  the  cost  to 
integrate  systems  together,  decreases  the  time  to 
integrate  systems,  and  so  lowers  the  cost  to  develop 
new  systems.  As  the  credibility  of  distributed  test 
matures  throughout  the  T&E  community,  more  and 
more  program  managers  will  incorporate  distributed 
test  into  their  test  strategies  and  documentation.  Early 
testing  in  the  intended  operational  environment  will 
become  commonplace.  Compliance  with  interopera¬ 
bility,  net-ready  KPPs,  and  measures  of  operational 
effectiveness  will  increase  the  acquisition  pipeline’s 
effectiveness,  as  well  as  increase  warfighting  systems’ 
overall  combat  capability;  this  means  warfighters  get  a 
better  product — cheaper  and  quicker!  The  joint 
infrastructure  needed  for  distributed  test  is  now  in 
place,  is  operational,  and  is  available  to  all  JMETC’s 
customers.  I  invite  the  T&E  community,  and  program 
managers  specifically,  to  contact  the  JMETC  team  for 
more  information  on  how  to  use  distributed  test  to 
support  your  program  and  T&E  events.  That  is  why 
JMETC  exists.  You  can  contact  us  directly  or  go  to  the 
JMETC  Web  site  at  www.jmetc.org.  □ 
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